1. Field of the Invention
The present invention relates to an image processing apparatus that processes an image of a subject and estimates a spectral characteristic of the subject, and to a computer program product.
2. Description of the Related Art
A spectral transmittance is one type of physical quantity representing physical characteristics unique to a subject. The spectrum transmittance is a physical amount that represents a ratio of transmitted light to incident light at each wavelength. Different from color information depending on variations in illumination light, such as R, G, and B values, the spectral transmittance is information unique to an object and it does not vary due to external influence. For this reason, the spectral transmittance is used in various fields as information for reproducing the color of the subject. For example, in the field of pathological diagnosis using a body tissue specimen, particularly, a pathological specimen, technology for estimating a spectral transmittance is used for analyzing an image of the specimen.
Pathological diagnosis is widely performed in which a block specimen extirpated from an internal organ or a pathological specimen obtained by needle biopsy is sliced to few microns, and the sliced specimen is then enlarged with a microscope for observation to have various findings. Particularly, observation in which transmitted light is observed employing an optical microscope is one of the most common observation methods, because it requires equipments that are relatively low-cost and easily handled and it is a time-honored method. In the observation, because the sliced specimen is almost clear and colorless and it does not absorb or scatter light, the sliced specimen is usually stained with pigment before observation.
Various staining methods up to 100 types or more have been proposed. Particularly for pathological specimens, haematoxylin-eosin staining (hereinafter, “H&E staining”) employing haematoxylin for violet and eosin for red are normally used as pigment.
Haematoxylin is a natural substance sampled from plants, and it is not stainable. However, haematin, which is oxidized haematoxylin, is a basophilic pigment that is bound to a negatively-charged material. Because deoxyribo nucleic acid (DNA) contained in a cell nucleus is negatively charged because of the phosphate group contained as a constituent, DNA is bound to haematin and stained to violet. Although, as described above, not haematoxylin but haematin, which is oxidized haematoxylin, is stainable, haematoxylin is usually used as the name of the pigment. Therefore, haematoxylin is used as the pigment below. On the other hand, eosin is an acidophil pigment that is bound to a positively-charged material. The pH environment influences whether amino acid or protein is negatively or positively charged, and they tend to be positively charged in an acidic state. For this reason, an eosin solution added with acetic acid is used in some cases. Proteins contained in cytoplasm are bound to eosin and stained from red to light red.
In a specimen stained through the HaG staining (stained specimen), cell nuclei and bone tissues are stained to violet, while cytoplasm, connective tissues, and erythrocytes are stained to red, so that they are easily recognizable. As a result, an observer can see the sizes of and positional relationship between elements constituting tissues including cell nuclei, thereby determining the state of the stained specimen morphologically.
The stained specimen can be visually observed by an observer. Alternatively, the stained specimen can be observed in a manner that a multiband image of the stained specimen is picked up and the multiband image is displayed on a display screen of an external device. When the image is displayed on the display screen, processing for estimating a spectral transmittance of each point on the specimen from the multiband image, processing for estimating the amount of pigment with which the specimen is stained based on the estimated spectrum transmittance, and processing for correcting the color of the image based on the estimated amount of pigment. Accordingly, the variance in characteristics of cameras and stain condition is corrected, so that an RGB image of the specimen to be displayed is synthesized. FIG. 13 is a graph of an example of the synthesized RGB image. If the amount of pigment is appropriately estimated, an image of a specimen stained to thicker colors and light colors can be corrected to an image in a color equivalent to that of an appropriately-stained specimen. For this reason, estimating a spectral transmittance of a stained specimen with high accuracy leads to highly accurately estimating the fixed amount of pigment of the stained specimen and correcting variance in staining.
Methods of estimating a spectral transmittance at each point on a specimen from multiband images of the specimen include an estimating method employing analysis of primary component (for example, see “Development of support systems for pathology using spectral transmittance—The quantification method of stain conditions”, Proceeding of SPIE, Vol. 4684, 2002, pp. 1516-1523), or an estimating method employing Wiener estimation (for example, see “Color Correction of Pathological Images Based on Dye Amount Quantification”, OPTICAL REVIEW, Vol. 12, No. 4, 2005, pp. 293-300) The Wiener estimation is widely known as a linear filtering method of estimating an original signal from an observed signal with noises in which errors are minimized in consideration of statistical characteristic of a subject to be observed and characteristics of observed noise. Because some noise is contained in a signal from a camera, the Wiener estimation is significantly useful as a method of estimating an original signal.
The method of estimating a spectral transmittance at each point on a specimen from multiband images of the specimen, employing the Wiener estimation, is explained below.
First, a multiband image of the specimen is picked up by, for example, the technology disclosed in Japanese Patent Application Laid-open No. H7-120324 employing a frame sequential method in which 16 bandpass filters are rotated with a filter wheel to switch the bandpass filters. Accordingly, the multiband image with pixel values of 16 bands at each point on the specimen is obtained. Pigment spreads three-dimensionally in the stained specimen to be observed originally. In a normal observation system in which transmitted light observation is performed, however, the specimen cannot be taken as a three dimensional image, and it is observed as a two-dimensional image obtained by projecting illumination light having transmitted through the specimen on an imaging device of a camera. Therefore, each point denotes each point on the specimen that corresponds to each pixel of the image projected on the imaging device.
Regarding a position x on the multiband image, the following Equation (1) based on a response system of the camera is satisfied between a pixel value g(x, b) and a spectral transmittance t(x, λ) at a corresponding point on the specimen.g(x,b)=∫f(b,λ)s(λ)e(λ)t(x,λ)dλ+n(b)  (1)where λ is a wavelength, f(b, λ) is a spectral transmittance of a b-th filter, s(λ) is a spectral sensitivity, e(λ) is a spectral radiation characteristic, n(b) is an observed noise of the band b. The band b is a serial number identifying the band and is an integer satisfying 1≦b≦16.
In actual calculation, the following Equation (2) obtained by a discrete method in a wavelength direction is used.G(x)=FSET(x)+N   (2)
When the number of sample points in the wavelength direction is D and the number of bands is B (B=16 in this case), G(x) is a matrix of B rows and 1 column corresponding to the pixel value g(x, b) at the position x. Similarly, T(x) is a matrix of D rows and one column corresponding to t(x, λ), F is a matrix of B rows and D columns corresponding to f(b, λ). On the other hand, S is a diagonal matrix of D rows and D columns, and a diagonal element corresponds to s(λ) Similarly, E is a diagonal matrix of D rows and D columns and a diagonal element corresponds to e(λ). N is a matrix of B rows and one column corresponding to n(b). In Equation (2), because formulas about a plurality of bands are aggregated using the matrices, the variable b representing the band is not explicitly described. An integral of the wavelength λ is replaced by a product of the matrices.
To simplify the expression, a matrix H defined by the following Equation (3) is introduced. H is also referred to as a system matrix.H=FSE   (3)
Subsequently, a spectral transmittance at each point on the specimen is estimated from the multiband image, using the Wiener estimation. An estimated value {circumflex over (T)}(x) can be calculated from the following Equation (4){circumflex over (T)}(x)=WG(x)   (4)
W in Equation (4) is represented by the following Equation (5), and is referred to as “Wiener estimated matrix” or “estimation operator used for Wiener estimation”. In the following explanation, W is simply referred to as “estimation operator”.W=RSSHt(HRSSHt+RNN)−1   (5)where Ht represents a transposed matrix of H, (HRSSHt+RNN)−1 represents an inverse matrix of HRSSHt+RNN, and RSS is an matrix of the row D and the column D that represents an autocorrelation matrix of the spectral transmittance of the specimen. RNN is a matrix of B rows and B columns that represents an autocorrelation matrix of noise of the camera to be used to pick up an image. The estimation operator consists of the system matrix H, a term RSS representing a statistical characteristic, and a term RNN representing a characteristic of the observed noise. Highly accurately representing each characteristic leads to improvement of accuracy in estimating the spectral transmittance.